Search for: All records

This study explores the potential of the large language model GPT-4 as an automated tool for qualitative data analysis by educational researchers, exploring which techniques are most successful for different types of constructs. Specifically, we assess three different prompt engineering strategies — Zero-shot, Few-shot, and Few-shot with contextual information — as well as the use of embeddings. We do so in the context of qualitatively coding three distinct educational datasets: Algebra I semi-personalized tutoring session transcripts, student observations in a game-based learning environment, and debugging behaviours in an introductory programming course. We evaluated the performance of each approach based on its inter-rater agreement with human coders and explored how different methods vary in effectiveness depending on a construct’s degree of clarity, concreteness, objectivity, granularity, and specificity. Our findings suggest that while GPT-4 can code a broad range of constructs, no single method consistently outperforms the others, and the selection of a particular method should be tailored to the specific properties of the construct and context being analyzed. We also found that GPT-4 has the most difficulty with the same constructs than human coders find more difficult to reach inter-rater reliability on. 

Abstract This study investigates student learning and interest within the context of a single-player, open-world game designed for microbiology inquiry. The game immerses players in the role of investigative scientists tasked with diagnosing a mysterious illness on a remote island. Ordered Network Analysis (ONA) was combined with clustering techniques to analyze in-game actions (i.e., interactions with non-playable characters, exploration, and utilization of in-game educational tools) allowing us to construct student archetypes based on the behavioral patterns of 122 middle schoolers. The analysis identified four distinct clusters of students with varying engagement patterns—two showing apparent patterns of engagement and two showing apparent patterns of disengagement. The study contributes insights into tailoring educational game designs to address disengaged or ineffective behaviors, enhancing the efficacy of game-based learning experiences. 

PurposeThere is a need for precollege learning designs that empower youth to be epistemic agents in contexts that intersect burgeoning areas of computing, big data and social media. The purpose of this study is to explore how “sandbox” or open-inquiry data science with social media supports learning. Design/methodology/approachThis paper offers vignettes from an illustrative youth study case that highlights the pedagogical prospects and obstacles tied to designing for open-ended inquiry with computational data science to access or “scrape” Twitter/X. The youth case showcases how social media can be taken up productively and in ways that facilitate epistemological agency, an approach where individuals actively shape understanding and knowledge-creation processes, highlighting the potentially transformative impact this approach might have in empowering learners to engage productively. FindingsThe authors identify three key affordances for learning that emerged from the illustrative case: (1) flexible opportunities for content-specific domain mastery, (2) situated inquiry that embodies next-generation science practices and (3) embedded computational skill development. The authors discuss these findings in relation to contemporary education needs to broaden participation in data science and computing. Originality/valueTo address challenges in current data science education associated with supporting sustained and productive engagement in computing-based data science, the authors leverage a “sandbox” approach – an original pedagogical framework to support open inquiry with precollege groups. The authors demonstrate how “big data” drawn from social media with high school-aged youth supports learning designs and outcomes by emphasizing learner interests and authentic practice. 

Abstract BackgroundSTEM minority participation programs have been widely implemented in higher education with the goal of diversifying the global STEM workforce. Informed by research highlighting the potential of targeted exploration of STEM roles and reflection on theselfin relation to STEM (identity exploration), this work examines how engagement in a government funded STEM minority participation program shaped these processes in current students and program alumni. ResultsEpistemic network analysis (ENA) was used to visualize conceptual connections between identity themes that emerged from interviews with present and past program participants. Network models were developed for current students and alumni for cross-group comparisons. Differences were found in how participants at different stages of their careers enact and describe their identity exploration processes. Summative network models highlighted how students discussed action-taking (sometimes through participation in STEM minority program initiatives) as they explored less-certain possible future STEM roles, while alumni integrated more diverse and holistic facets of their identities when conceptualizing their futures. To close the interpretive loop, a qualitative interpretation of interview discourse was used to give context to network patterns. ConclusionsResults highlight the differences between novices’ and professionals’ conceptualizations of their future selves and illustrate how minoritized individuals describe their long-term patterns of identity exploration related to STEM majors and careers. Implications for future STEM identity research and practice, including higher education programming as a tool to support students’ STEM identity exploration processes, are discussed.